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The problem of optium reception of M-ary Gaussian signals in Gaussian 
noise is to specify, in terms of the observable waveform, a scheme for deciding 
among M alternative mean and covariance functions with minimum error 
probability. Although much literature on the problem exists, a mathemati- 
cally rigorous solution has yet to appear. By formulating the problem as 
optimum discrimination of M Gaussian measures in function space in- 
duced by the mean and covariance functions, this paper presents such a 

solution. 

Let m k (t) and r k (s,t), k = 1, ■■■ , M, be the alternative mean and 
covariance functions of the Gaussian signal, and let m (t) and r (s,t) be 
the mean and covariance functions of the Gaussian noise. If, for each k = 
1, • • • , M, the integral equations, 

J r (s,t)g k (s) ds = m k (t) 

ff r (s,u)h k (u,v)[r (v,t) + r k (v,t)} du dv = r k (s,l), 

admit a square-integrable solution g k (t) and a symmetric, square-integrable 
solution h k (s,t), then the following decision scheme is optimum: given an 
observable waveform x (I ) , 

choose m k (t) and r k (s,t) if h(x) is the largest among all Ij(x), 

j _ 1, ■ • • , M, 

where l k is defined by 

/,(. r ) = I ff x(s)h k (s,t).v(t) dsdt + f x(t)f k (t) dt + c* 

in which 
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and 
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fM = gM - f h k (s,t)[m (s) + m k (s)] ds 

and Ck is a constant determined by the mean and covariance functions 
m (t), m k it), r Q (s,t) and r* (s,t) as well as the a priori probability associated 
with m k {t) and r k (s,t). 

The first section introduces and defines the problem and the second pre- 
sents the solution with pertinent discussions while a precise mathematical 
treatment is left to the appendix. 

I. INTRODUCTION 

Before we formulate the general problem of optimum reception of 
M-ary signals in noise, let us review a simplified version of the classical 
problem: optimum reception of M-ary sure signals in Gaussian noise. 
Suppose there are M sure signals m k (t), k = 1, • • • , M, with a priori 

probabilities a k , < a* < 1 and J2"=i a* = 1, for transmission. The 
received waveform x (t ) consists of one of these M signals and an addi- 
tive Gaussian noise n(t), i.e. 

x(t) = m k {t) + n(t). 

In order to simplify the problem, we "represent" the signals and noise 
by certain finite sequences m ki , • • • , m kn , k — 1, • • • , M, and Wi , ■ • • , 
n n respectively.* Then the representing sequence Xi, ••• , x n of the 
received waveform is given by 

Xi = m ki + rii, i = 1, •• ■ , n. (1) 

It is assumed that the signal sequences are linearly independent vectors 
in an n-dimensional space R n while the elements, ni , ■ • • , n„ of the 
noise sequence are statistically independent, identically distributed 
Gaussian variables with mean zero and variance one. The task of the 
receiver is to observe the received sequence X\ , • • • , x n and to decide 
which one of M signal sequences must have been transmitted. For each 
possible erroneous decision, there is an associated probability, and the 
average of all these probabilities weighted by the a priori probabilities 
is the so-called (average) error probability. Then, the problem of 
optimum reception in this simplified form is to specify in terms of the 
observable sequence X\ , • • • , x n a scheme for choosing the value of the 
index A; such that the error probability is minimum over all possible 
decision schemes. 



* These sequences may be regarded as the sample values of the waveforms or 
the Fourier coefficients of certain orthonormal expansions of the waveforms. 
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Now, from the assumption concerning the noise sequence, the joint 
probability density function of Wi , • • • , n„ is 



Po(vi , 



= (2x 



>-(n/2) 



exp 



-m- 



(2) 



Hence, through the use of the relation (1), the joint density function of 
X\ , • ■ • , x„ becomes 



P*( 



Vl 



,v„)= (2tt) 



-(n/2) 



cxp 






(3) 



Note that since the observable sequence is a point in R„ , a decision 
scheme in terms of it is equivalent to a division of R„ into M non-over- 
lapping regions A x , • • • , A, M so that k = j is chosen if (.Ti , • • • , x„) 
belongs to A, . Then, the error probability associated with such a 
division (Ai , • ■ • , Am) is given by 

P e = 1 — £ a k I Vk(f\ , • ■ ■ , v n ) dui • • ■ dv„ , 



which can be rewritten as 



p.- i- j \t «. 

J r„ L*=i 



(l/! , • • • , V n )<XkVk(v\ , 



,^n) 



di»i 



rfv,, , (4) 



where x'\ h is the indicator function of the set A k , k = 1, • • • , M. Clearly, 
P e is minimum over all divisions if the integrand in the above is maxi- 
mum at every point (vi , • • • , v n ) in /?„ . This can be achieved by 
arranging a division (Ai , • • • , Am) so that, in the region A k , the a k p k 
with the same index is the largest among all avp\ , • • • , a M p M ■ Namely, 
the region with index k consists of a set of points at which a k p k is the 
largest. If more than one a k p k is the largest at the same point, that 
point is to be assigned to the region whose index is the smallest of all 
those fc's so that all the regions remain disjoint. That is, a division 
(Si, • • • , S M ) of R„ specified by 



S k = {(»,, 



, v n )'- a k p k (vi , 
a k p k (vi , 



, v n ) > a/pjivi , 

, V a ) ^ CtJPjfa , 



Vn), ) 


J < A, 


Vn), 


J > k j 



(5) 



is the division which minimizes P e . Or the optimum decision scheme is 
the following: 
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Given an observable sequence (rci , • • • , x n ), choose the mini- 
mum value of k for which a k pk (#1 , • • • , x n ) is maximum as a 
function of k.* 
Now by expanding the exponent in (3), 

<XkPk(Xl , • • • , X n ) = Ok exp ( - - X X i + S • X *' m *' "o£ mk *) • 

Since the first sum in the bracket is common to all a*p* , if we put 

II 1 B 

Ik = 2 x t m k i -;E m fci 2 + log a k , (6) 

then the optimum decision scheme is equivalent to choosing the mini- 
mum value of k for which Ik is maximum. 

In the special case of "equi-probable and equi-energetic" transmitted 
signal sequences, i.e. 

n n 

a, : = • • • =«m and ^ Wi/ = • • • = ^ m Mt - 2 , 

i=l i=l 

/fc can be effectively replaced by its first term, i.e. ^?=i x<m,ki ■ In other 
words, the optimum decision scheme in this case consists in performing 
correlation of the observable sequence with M signal sequences and 
choosing the smallest of the k values corresponding to the largest 
correlation sums.f 

In the general problem of optimum reception of M-ary Gaussian 
signals in Gaussian noise, the observable waveform (received waveform) 
x(t) is expressed by 

x(t) = y k (t) +n(t) 

where yk(t) is one of M possible Gaussian signals which are character- 
ized by mean and covariance functions just as n(t) is. We assume that 
each Gaussian signal is statistically independent of the noise and it 
cannot be detected "perfectly" in the presence of this noise. J Again, the 
task of the receiver is to observe the waveform x(t) for a finite time, 
say ^ t ^ 1, and to decide which one of M Gaussian signals must have 
been received. Then, by defining the error probability as before, the 
problem of optimum reception becomes that of specifying, in terms of the 

* That is, suppose for a given (xi , • • • , x„), a*p*(xi , • • • , x„) as a function 
of k assumes its maximum at k = ki , • • • , kj where k i < • • • < kj . Then, we 
choose k = ki . 

t For classical references, see Refs. 1 and 2. 

j This is the assumption of "non-singular detection". A necessary and suffi- 
cient condition of non-singular detection is given by (13) in Appendix. 
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observable waveform x(l), a .scheme for choosing the index k such that 
the error probability is minimum over all possible decision schemes. 

One mathematical idealization of the above problem is the following: 
Let \yt , ^ / ^ 1| be a Gaussian process whose mean and covariance 
functions are one of M possible pairs of m k (t), ^ t ^ 1, and n(s,t), 
^ s, t ^ 1, k = 1, ■ • • , M , where m k (t) and r k (s,t) are assumed to be 
continuous.* Similarly, let \n, , ^ / ^ 1} be a Gaussian process whose 
mean and covariance functions are m (t), ^ < ^ 1, and r (s,t), ^ 
s,i ^ 1, where m a (t) is assumed to be continuous while ro(s,t) is positive- 
definite as well as continuous. It is further assumed that \yt, % t St 1) 
and j/i, , ^ < ^ 1) are mutually independent for every A- = 1, • • • , M. 
Now define a new process [x t , ^ t ^ 1} by .r, = ifi -\- n, . Then, from 
the mutual independence assumption, the mean and covariance func- 
tions of [xt ,0 ^ < ^ 1) are one of M possible pairs of m (t) + m k {t) 
and r (s,0 + r*(«,0, fc = 1, • • • , M . Let P k , k = 1, • •■■ , M, be the 
Gaussian (probability) measure corresponding to the pair m (t) + 
m k (t) and r (s,t) + r k (s,t), and let P be the one corresponding to m (l) 
and r (s,0- It is assumed that m (t), ?n k (t), r (s,t) and r k (s,t) are such 
that the two measures P and P k are equivalent, i.e. P = P k , fc = 1, 
• • • , M.\ Denote by H k and a k , k = 1, • • • , M, the hypothesis and 
a priori probability that m k {t) and r k (s,t) are the pertinent mean and 
covariance functions of \y t , ^ t ^ lj. Let #(<) be the sample function 
of [xt , ^ < ^ 1). Then, specification of the decision scheme amounts 
to dividing the space Q of all .sample functions x(t) into M disjoint sets, 
Ai , • • • , A M , so that, if x(t) £ A* , then H k is to be chosen. Moreover, 
the error probability associated with such a division (or a decision 
scheme) is given by 

Pc= 1 - 2>A-P*(A t ). (7) 

Thus, the problem of optimum reception is to specify in terms of x (t) 
such a division of 12 that its associated error probability is minimum over 
all possible divisions. 

Unlike the previous simple case, the sample (the observable) in this 
general case is the sample function x(t) of the Gaussian process \x t , ^ 
t ^ lj instead of the sample sequence X\ , • • • , x n of the finite sequence 
of Gaussian variables. Thus, the sample space which is to be divided 
into M non-overlapping regions, is the function space Q, instead of the 
n-dimensional sequence space R„ . Hence, we no longer have at our 

* Note: n(8,t) = E k \ (x, - m k (s))(xj - m k (t))\,k = 1, • • • , M. 
f This corresponds lo the assumption of non-singular detection. 
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disposal the joint density functions pic(vi , • • • , v n ), k = 1, • • • , M, 
through which the optimum decision scheme is constructed. Neverthe- 
less, there exists a certain generalization to this basic approach. Note 
from (2) that po(vi , • ■ ■ , v„) > 0, — °° < vt< <», i «= 1, • • • , n. Hence, 
(4) can be rewritten as 



l\ = 






X dvi • ■ ■ dv n . 



Then, the optimum division of R„ is specified in terms of a k [-p k {xi , • • • , 
x„)/p (xi , • • • , x H )] instead of a k p k (xi , • • • , .r„), though the two are 
obviously equivalent. Now, in the general case where the sample space 
is fl instead of R„ , the likelihood ratio p k (xi , • • • , x„)/p (xi , • • • , .r„) 
is replaced by its generalized version dPk/dPa , the Radon-Nikodym 
derivative (of P k with respect to P ), which is a function of x(t), and 
Po(vi , ■ • • , v n )dvi • ■ ■ dv n is replaced by dP ■ Thus, the error probability 
in the general case can be expressed as 



P.- I - / l£xA»«*fr(*) 






dP t 



dPo(x), 



where (Ai , • • • , A«) is a nonoverlapping division of J2. Then an opti- 
mum division of 0, which is analogous to (5), can be specified by 
a K (dP k /dP ), k = 1, • • • , M, and d,P k /dP a can in turn be expressed in 
terms of certain functionals of x(t). 

II. SUMMARY OF MAIN RESULTS AND DISCUSSIONS 

The foundation for solution of the general problem stated in the 
preceding section consists of the two following facts: 3 4 

(i) If two Gaussian measures P and P k are equivalent for each k = 
1, • • • , M, then there exists random variables dP k /dP so that the 
optimum division (Si , ■ ■ ■ , S M ) of the sample space ft can be specified 
by 



S k = <x(t): a k ^ (x) > a JT ^(x),j < k; 

dP k . s . rfPy M . v , 

ak dP (x) = a idF {x) > J>k ;- 

(it) If the integral equations 
i 
r (s,t)g k (s) ds = m k (l), 5 t £ 1, (9) 



.( 



RECEPTION OF M-ABT GAUSSIAN SIGNALS 2193 

and 

nro(s,u)hk(u,v)[r (v,t) + r k (v,t)] du dv = r k (s,t), 
(10) 

^ 8,1 ^ 1, 

have a square-integrable solution g k (t) and a symmetric, square-inte- 
grable solution h k (s,t) respectively,* then 



dP K *' 



fit exp - J I [x(s) — m (s) — m k (s)]h k (s,t) 

■[x(t) - mo(t) - mM] ds dt (11) 

+ f [x(t) - mM - hmM]gM dt\ , 

for almost all sample functions under all hypotheses H k , k = 1, • • • , M, 
where fa' 1 = II "=i X» (W and X, a> > 0, i = 1,2, • • ■ , are the eigenvalues 
of the operators # _1 (Ro + fl*)/2o _i and R and R k are integral operators 
whose kernels are r (s,t) and r k (s,t) respectively. 

Then, upon combination of (i) and (tt), the optimum decision 
scheme can be specified as follows: 

Choose the minimum value of k for which /* is maximum 

where 

h = I ( [ x(s)h k (s,t)x(t) dsdt+ f x(t)f k (l) dt 

2j%J \ t ° (12) 

+ o / / [ m '»( s ) + ™> k (s)] h(s,t) [?no(0 + m k (t)]dsdt 

L >>o •'0 

- / [mo(0 + i w*(0] flt(0 * + %«*M 

J o 

in which /*(0, fc = 1, • • • , M, are defined by 

ft(t) = fift(0 - / ft*(«^)[«o(a) + ro*(s)] ds, g ^ 1. 

Needless to say, the condition for the above decision scheme to be 
optimum is the existence of such g k (t) and h k (s,t), k = 1, ■ ■ • , M, as 
described in (it). It should be remarked that the existence of g k (t) 



* If such solutions exist, they are necessarily unique. It should be remarked 
that square-integrability of h(s,t) is in the sense of 

-I ^i 

h°-(s,t)ds dt < °° . 
'o Jo 



J t\ Jo 
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and hk(s,t) implies that Po and Pk are equivalent for each k. That is, if 
these Qk(t) and hk(s,t) exist, then none of the Gaussian signals yt(t) 
can be detected perfectly in the presence of noise n(t). 

Physical interpretation of the above optimum decision scheme is 
straightforward, at least in principle. Suppose given m (t), m,k{t), r (s,t) 
and n(s,t), k = 1, • • • , M, are such that the integral equations (9) and 
(10) admit a square-integrable solution gt(t) and a symmetric, square- 
integrable solution hk(s,t), then the optimum decision scheme consists 
in performing the single and the double integrals involving the received 
waveform x(t) as specified by (12), and adding to these integrals the 
predetermined constants, the last three terms of (12), and finally choos- 
ing the minimum value of k for which the sum of these integrals and the 
constants is maximum. 

It is instructive to consider the following two special cases: 

Case 1 , m (t) = 0, r k (s,t) = 0, k = 1, ■ • • , M. 

This is the case of "M-ary sure signals in noise", a simplified version 
of which has already been discussed in the introduction. Here the integral 
equation (10) always has a symmetric, square-integrable solution for 
each k = 1, • • ■ , M, namely, the trivial solution: 

h k (s,t) s0. 

Furthermore, X, a ' = 1, i = 1,2, ■ ■ ■ ; k = 1, ■ ■ ■ , M. Thus, 

ft = 1, k = 1, ■■■ , M. 

Hence, h of (12) is reduced to 

h' = / x(t)g k (t) — s / m k (t)gk(t) dt + log a* , 

•'o ^ Jo 

provided the square-integrable solutions g k (t), k = 1, ••• , M, exist 
for the integral equations (9). Note that I* is the function-space 
counterpart to l k of (6) in the sequence-space case. With additional 
conditions that 

ai = • • • = a M and / mi(t)gi(t) dt = • • • = / m M (t)g M (t) dt,f 

the optimum decision scheme is reduced to choosing the minimum value 



* The form of // agrees with the result formally obtained in Ref. 5. 

f For example, choose nik(l) = Vffi- ^*(0, k = 1, • • • , M, where <n ^ <r 2 ^ ■•• , 
and >p\(l), yp-'(l), ■■■ are the eigenvalues and the orthonormalized eigenfunctions 
of tf . 
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of k for which the "correlation integral" 

x(t)g k (t) dt 



I 



is maximum. 

Case 2 m (t) = 0, m k (t) = 0, k = 1, • • • , M. 

This is the case commonly termed as "M-ary Gaussian signals in 
noise". Here, the integral equation (9) always has a square-in tegrable 
solution, namely, the trivial solution: 

g k (t) =0, fc = 1, ••• ,M. 
Hence, I k is reduced to 

/," = i / ( x (s)h(s,t)x(t) ds dt + log a k p k \ 
Z Jo Jo 

Thus, the optimum decision scheme consists in choosing the minimum 
value of k for which I k " is maximum, provided that the symmetric;, 
square-integrable solutions h k (s,t), k = 1, • • • , M, of (10) exist. 
(Remark ) 

It is interesting to note that formal substitution of n(s,t) = 8(s — t) 
into (10) yields the result which is consistent with those obtained 
previously by Price. 

APPENDIX 

Mathematical Supplement 

In the preceding, mathematical precision has been somewhat com- 
promised for intuitive appeal. It is the purpose of this appendix to 
clarify the content of the preceding sections by supplying a brief mathe- 
matical summary with pertinent remarks. 

Let $2 be the space of all real- valued functions on [0,1] and let Po and 
P k , k = 1, • • • , M, be the Gaussian measures induced by m and r () 
and by m + m k and n + n respectively on a <r-field generated by the 
class of all intervals in £2, where w and m k , k = 1, • • • , M, are real- 
valued, continuous functions on [0,1], while r n and r k , k = 1, • • • , M, 
are real-valued, symmetric, positive-definite, continuous functions on 
[0,1] X [0,1].* Then, without loss of generality, there exists a real, 

* r k , k = 1, • • • , M, can be only nonnegative-definite. 
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.separable (with respect to P ) and measurable process {.v, ,0 ^ t ^ 1}.* 
Let 03 be the minimal tr-field with respect to which every x t , I (j [0,1], 
is measurable, and P and P k , k = 1, • • • , M, be the restrictions on (B 
of P and Pa- respectively. We assume that Po = Pk , k = 1, • ■ • , M, 
which immediately implies that P,: = P, ; ?'J = 1, • • • , Af, and {x t , ^ 
/ ^ 1J is separable with respect to P* , k = 1, • • • , M , also. For a neces- 
sary and sufficient condition for P = Pk , we cite the following:! 

lim tr [R M (R ln) + R k M ) _I - 27 + (fl ( " } + «* <B) ) (^o (n) ) -1 ] < * , 

limtr (Ro^y'M^ < «., (13) 

where flo ( *\ ft* (n) and M fc (n) , k = 1, • • • , M, are w X n-matrices defined 
by 

(flo ( '"),:y = r (t>,tj), 

(7?, ( "% = nit^tj), i,j = 1, ••■ ,n, 

(M* (B) ),-,- = m* &)«*&), 

and ^ , i = 1, • • ■ , ra, arc a finite subset of any sequence dense in [0,1]. 
Now the two fundamental facts for solution are: 

(0 Let dPk/dPo ,1c = 1, • ■ • , M, be the Radon-Nikodym derivatives 
of P k with respect to P , and let (Si , • • ■ , S M ) be the partition of fi 
defined by 

„ / . dP k , , rfPy , , . . \ 

n{.:*g(.)^^g(-u>*}- 

Then, for any partition (Ai , • • • , A^) of ft, 

Ea*P*(&) ^ E«,P fc (A*). 



(«'i) For each fc = 1, • • • , M, if the integral equations (9) and (10) 
admit a square-integrable solution g k and a solution hk satisfying 

h k (s,t) = h k (t,s), / / h k 2 (s,t) ds dt < », 

•Ml ■'(I 



* For a detailed justification, see Ref. 4. 
t See Ref. 4. 
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then 

(a) such solutions gt and h k are unique, 

(b) P ■ P k , k = 1, • • • , M, 

(c) there exist eigenvalues X, a> > 0, i = 1, 2, • • • , of 

/?o" 5 (/?o + Rk)Ro~ h 
such that n^=> x - a ' converges to ft"" 1 , < ft < x, and 

^* = ft* exp |i f f [.r, - w„(s) - m k (s)]h k (sMx t - m a (t) - m k (t)] ds dt 

(II |_2 J o J o 



Jo 



a.e. (P„). 



Hence, specification of *S* , k = 1, • • • , M, is obtained by combining 
(i) and (m), and the hypothesis in (ii) is the condition that such a 
partition (Si , • • • , S M ) exists and minimizes (7). 
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